Continuous extremal optimization for Lennard-Jones Clusters 
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In this paper, we explore a general-purpose heuristic algorithm for finding high-quality solutions 
to continuous optimization problems. The method, called continuous extremal optimization(CEO), 
can be considered as an extension of extremal optimization(EO) and is consisted of two components, 
one is with responsibility for global searching and the other is with responsibility for local searching. 
With only one adjustable parameter, the CEO's performance proves competitive with more elaborate 
stochastic optimization procedures. We demonstrate it on a well known continuous optimization 
problem: the Lennerd- Jones clusters optimization problem. 
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I. INTRODUCTION 

The optimization of system with many degrees of free- 
dom with respect to some cost function is a frequently 
encountered task in physics and beyond. One special 
class of algorithms used for finding the high-quality so- 
lutions to those NP-hard optimization problems is the 
so-called nature inspired algorithms, including simulated 
annealing (SA)'l','^, genetic algorithms(GA)0, 0, Q, ge- 
netic programming (GP) 6J, and so on. 

In recent years, a novel nature inspired algo- 
rithm named extremal optimization(EO) is proposed by 
Boettcher and Percus[3 S H 113. 113, which is very 
sententious and competitive comparing with some well 
known algorithms like SA, GA, GP etc.. To make the 
underlying mechanism of EO more concrete, let's focus 
on the natural selection of biological system. In nature, 
highly specialized, complex structure often emerge when 
their most inefficient elements are selectively driven to 
extinction. For example, evolution progresses by select- 
ing against the few most poorly adapted species, rather 
than by expressly breeding those species best adapted to 
their environment. The principle that the least fit ele- 
ments are progressively eliminated has been applied suc- 
cessfully in the Bak-Sneppen model0,0, where each 
individual corresponding a certain species is character- 
ized by a fitness value, and the least fit one with smallest 
fitness value and its closest dependent species are succes- 
sively selected for adaptive changes. The extremal opti- 
mization algorithm draws upon the Bak-Sneppen mech- 
anism, yielding a dynamic optimization procedure free of 
selection parameters. 
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Here we consider a general optimization problem, 
where the system consists of N elements, and we wish 
to minimize the cost function C{S) depending on the 
system configuration S . The EO algorithm proceeds as 
follows: 

(1) Choose an initial configuration S of the system at 
will; set S'best := S. 

(2) Evaluate the fitness value fi for each individual i and 
rank each individual according to its fitness value so as 
to the least fit one is in the top. Use ki to denote the 
individual i's rank, clearly, the least fit one is of rank 1. 
Choose one individual j that will be changed with the 
probability P{kj), and then, only randomly change the 
state of J and keep all other individuals' state unaltered. 
Accept the new configuration 5" unconditionally S :— S', 
and if C{S) < C(S'best) then set 5best := S. 

(3) Repeat at step (2) as long as desired. 

(4) Return 5best and C(S'best)- 

The efficiency of EO algorithm is sensitive to the prob- 
ability function P{k). In basic EO, P(l) — 1 and for any 
k{2 < k < N), P{k) = 0. A more efhcient algorithm, the 
so-called r-EO, can be obtained through a slight modifi- 
cation from basic EO. In r-EO, P(k) ^ k~'^ where r > 0. 
Of course, aiming at idiographic optimization problems, 
one can design various forms of P{k) to improve the per- 
formance of basic EO. For example, Middleton has pro- 
posed the jaded extremal optimization(JEO) method for 
Ising spin glass system by reducing the probability of 
flipping previously selected spins, which remarkably im- 
proved the efficiency of EO . 

The previous studies indicate that EO algorithm can 
often outperform some far more complicated or finely 
tuned algorithm, such as SA or GA, on some famous NP- 
hard|l5j discrete optimization problems, including graph 
partitioning^, S llfij - traveUing salesman problem J^, 
three-coloring problem 0,0, findin g low est energy con- 
figuration for Iring spin glass svstem p^ IttI I19 | . and so 
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on. However, many practical problems can not be ab- 
stracted to discrete form, thus to investigate EO's ef- 
ficiency on continuous optimization problems 20] is not 
only of theoretic interest, but also of prominent practical 
worthiness. 

In this paper, a so-called continuous extremal op- 
timization(CEO) algorithm aiming at continuous opti- 
mization problem will be introduced, which can be con- 
sidered as a mixing algorithm consisting of two com- 
ponents, one is with responsibility for global searching 
and the other is with responsibility for local searching. 
With only one adjustable parameter, the CEO's perfor- 
mance proves competitive with more elaborate stochastic 
optimization procedures. We demonstrate it on a well 
known continuous optimization problem: the Lennerd- 
Jones(LJ) clusters optimization problem. 

This paper is organized as follows: in section 2, the 
LJ clusters optimization problem will be briefly intro- 
duced. In section 3, we will give the algorithm proceeds 
of CEO. Next, we give the computing results about the 
performance of CEO on LJ clusters optimization prob- 
lem. Finally, in section 5, the conclusion is drawn and 
the relevances of the CEO to the real-life problems are 
discussed. 



II. LENNERD-JONES CLUSTERS 
OPTIMIZATION PROBLEM 



Continuous optimization problem is ubiquitous in ma- 
terials science: many situation involve finding the struc- 
ture of clusters and the dependence of structure on size is 
particularly complex and intriguing. In practice, we usu- 
ally choose a potential function to take the most steady 
structure since it's considered to be in possession of the 
minima energy. However, in all but the simplest cases, 
these problem are complicated due to the presence of 
many local minima. Such problem is encountered in 
many area of science and engineering, for example, the 
notorious protein folding problem |2l|. 

As one of the simplest models that exhibits such be- 
havior |22| one may consider the problem of finding the 
ground-state structure of nanocluster of atoms interact- 
ing through a classical Lennerd- Jones pair potential, in 
reduced units given by 



(1) 



where r is the distance between two atoms. This po- 
tential has a single minimum at r = which is the 
equilibrium distance of two atoms. It can, of course, eas- 
ily be reduced to an arbitrary LJ-potential by a simple 
rescaling of length and energy units. The zth atom has 
energy 
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FIG. 1: The details of r-CEO for r e [1, 2]. Figure la shows 
the average energies obtained by CEO over 200 runs, and 
figure lb exhibits the success rate of hitting the global minima 
in 200 runs|2^. For both figure la and lb, the main plot and 
inset represent the case A'' = 40 and A*' = 30 respectively. One 
can find that, the best r corresponding lowest average energy 
and highest success rate is approximate to 1.5. 



and the total energy for N atoms is 

E=Y^E., (3) 

i 

The optimization task is to find the configuration with 
minimum total potential energy of a system of TV atoms, 
each pair interacting by potential of the form (1). Clearly, 
a trivial lower bound for the total energy is —NiN ~ l)/2, 
obtained when one assumes that all pairs are at their 
equilibrium separation. For A'^ = 2,3,4 the lower bound 
can actually be obtained in three-dimensional space, cor- 
responding respectively to a dimer, equilateral triangle. 
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and regular tetrahedron, with all interatomic distance 
equal to 1. However, from N = 5 onwards it is not possi- 
ble to place all the atoms simultaneously at the potential 
minimum of all others and the ground-state energy is 
strictly larger than the trivial lower bound. This system 
has been studied intensely _23] and is known to have an 
exponential increasing number of local minima, growing 
roughly as e'^-^e^+o os^v^ ^^^^^ ^ ^ ^A^^* which point 
there are already at least 988 minima [23 • If this scaling 
continues, more than 10^'*'^ local minima exist when N 
approach 100. 



III. CONTINUOUS EXTREMAL 
OPTIMIZATION 

The continuous extremal optimization algorithm is 
consisted of two components, one is the classical EO al- 
gorithm with responsibility for global searching, and the 
other is a certain local searching algorithm. We give the 
general form of CEO algorithm by way of the LJ clusters 
optimization problem as follows: 

(1) Choose an initial state of the system, where all 
the atoms are placed within a spherical container with 
radius (24 i25i| 



(a) 



radius ^ Vpl — h ;= 

2 UttV2 
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where Vf. — ■\/2 is the equilibrium distance and A'' denotes 
the number of atoms. Set the minimal energy iSmin = 0. 

(2) Use a certain local searching algorithm to find the 
local minimum from the current configuration of system. 
If the local minimal energy is lower than -Emin, then re- 
place Emin by the present local minimum. 

(3) Rank each atom according to its energy obtained by 
Equ.(2). Here, the atom who has highest energy is the 
least fit one and is arranged in the top of the queue. 
Choose one atom j that will be changed with the prob- 
ability P{kj) where kj denotes the rank of atom j, and 
then, only randomly change the coordinates of j and keep 
all other atoms' positions unaltered. Accept the new con- 
figuration unconditionally. 

(4) Repeat at step (2) as long as desired. 

(5) Return the minimal energy Elnin and the correspond- 
ing configuration. 

For an idiographic problem, one can attempt various 
local searching algorithms and pitch on the best one. 
In this paper, for the LJ clusters optimization problem, 
we choose limited memory BFGS method(LBFGS) qua 
the local searching algorithm. The BFGS method is an 
optimization technique based on quasi-Newton method 
proposed by Broyden, Fletcher, Goldfard and Shanno. 
LBFGS proposed by Liu and Nocedal|2^ |^ is espe- 
cially effective on problems involving a large number of 
variables. In this method, an approximation Hk to the 
inverse of the Hessian is obtained by applying M BFGS 
updates to a diagonal matrix Hq^ using information from 
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FIG. 2: The performance of CEO algorithm on LJ clusters 
optimization problem. In figure 2a, the red circles represent 
the average energies obtained by CEO over 200 runs, where 
the black squares represent the global minima. Figure 2b 
shows the success rate of hitting the global minima in 200 
runs, the inset is the success rate for > 50 that may be 
unclear in the main plot. 



the previous M steps. The number M determines the 
amount of storage required by the routine, which is spec- 
ified by the user, usually 3 < M < 7 and in our compu- 
tation M is fixed as 4. 



IV. COMPUTING RESULTS 

Similar to r-EO, we use r-CEO algorithm for the 
LJ clusters optimization problem, where the probability 
function of CEO is P{k) ^ k~'^ . Since there are ^ pairs 
of interactional atoms in a LJ cluster of size N , we require 
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FIG. 3: The average CPU time(ms) over 200 runs va the size 
of LJ clusters. In the log-log plot, the data can be well fitted 
by a straight line with slope 3.886 ± 0.008, which indicates 
that the increasing tendency of CPU time T vs cluster size is 
approximate to power-law form as T ~ jY'^ **®. 



aN^ updates where a is a constant and fixed as 100 in 
the following computation. In order to avoid falling into 
the same local minimum too many times, before run- 
ning LBFGS, we should make the system configuration 
far away from last local minimum, thus we run LBFGS 
every 20 time steps. That is to say, for a LJ cluster of 
size A'', the present algorithm runs EO lOOiV^ times and 
LBFGS 5A^2 tjjnes ^^^g^i 

We have carried out the r-CEO algorithm so many 
times for different t and N, and find that the algorithm 
performs better when r is in the interval [1,2]. In figure 1, 
we report the details for 1 < r < 2, where figure la shows 
the average energies obtained by CEO over 200 runs, and 
figure lb exhibits the success rate R of hitting the global 
minimaprll . For both figure la and lb, the main plot and 
inset represent the case = 40 and = 30 respectively. 
The readers should note that, although the difference of 
average energies between two different r is great in the 
plot, it is very very small in fact. For the case = 40, 
the best value of r is r = 1.6 corresponding the lowest 
average energy and highest success rate, however, the 
performance of CEO for r = 1.5 is almost the same as r = 
1.6 in this case but obviously better than r — 1.6 in the 
case A^ = 30. Therefore, in the following computation, 
we set r = 1.5. We have also compared the performance 
of CEO on larger LJ clusters for r = 1.5 and r = 1.6, the 
two cases are pretty much the same thing and t = 1.5 is 
a little better. 

We demonstrate that for all the LJ clusters of size A^ 
not more than 100, the global minima can be obtained 
by using CEO algorithm. In figure 2, we report the 
performance of CEO on LJ clusters optimization prob- 
lem according to 200 independent runs. In figure 2a, 
the red circles represent the average energies obtained 



by CEO over 200 runs, where the black squares repre- 
sent the global minima. One can see that the deviation 
from global minimum becomes greater and greater when 
the cluster size getting larger and larger, which indicates 
that for very large LJ cluster, CEO may be a poor al- 
gorithm. Figure 2b shows the success rate of hitting the 
global minima in 200 runs, the inset is the success rate 
for A' > 50 that may be unclear in the main plot. For 
both the case A^ = 95 and A^ = 100, the global optimal 
solutions appears only once in 200 runs. 

Although CEO is not a all-powerful algorithm and it 
may perform poorly for very large LJ clusters, we demon- 
strate that it is competitive or even superior over some 
more elaborate stochastic optimization procedures like 
SA'2^, GA''29'| in finding the most stable structure of LJ 
clusters with minimal energy. 

Finally, we investigate the average CPU time over 
200 runs vs the size of LJ clusters. The computations 
were carried out in a single Pentiumlll processor (1 GHZ). 
From figure 3, in the log-log plot, the data can be well fit- 
ted by a straight line with slope 3.326±0.008, which indi- 
cates that the increasing tendency of CPU time T vs clus- 
ter size is approximate to power-law form as T ~ ]\[3-SS6^ 
That means the CEO is a polynomial algorithm of order 
0(iV4). 



V. CONCLUSION AND DISCUSSION 

In this paper, we explore a general-purpose heuris- 
tic algorithm for finding high-quality solutions to con- 
tinuous optimization problems. The computing results 
indicate that this simple approach is competitive and 
sometimes can outperform some far more complicated 
or finely tuned nature inspired algorithm including sim- 
ulated annealing and genetic algorithm, on a well-known 
NP-hard continuous optimization problem for LJ clus- 
ters(see reference psl l29l| for comparison). According to 
EO's updating rule, it is clear that EO has very high abil- 
ity in global searching, thus to combine EO and a strong 
local searching algorithm may produce a high efficient 
algorithm for continuous optimization problems. 

Recently, several novel algorithms aiming at LJ clus- 
ters optimization problem have been prop osed, such 
as fast annealing evolutionary algorithm|25j. conforma- 
tional space annealing method 30] , adaptive immune 
optimization algorithm,31.j. cluster similarity checking 
method 32], and so forth. These algorithms consider 
more about the special information about LJ clusters 
and perform better than CEO. However, we have not 
found a compellent evidence indicating that there exists a 
general-purpose algorithm like SA or GA entirely prepon- 
derate over CEO on LJ cluster optimization problem. It 
is worthwhile to emphasize that, in this paper, we do not 
want to prove that the CEO is an all-powerful algorithm, 
even do not want to say that the CEO is a good choice 
for chemists on LJ cluster optimization problem since a 
general-purpose method often perform poorer than some 
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special methods aiming at an idiographic problem. The 
only thing we want to say is the CEO, an extension of 
nature inspired algorithm EO, is a competitive algorithm 
and needs more attention. 

Further more, to demonstrate the efficiency of CEO, 
much more experiments on various hard continuous op- 
timization problems should be achieved. 
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